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The Institute for Scholarship on Engineering Education (ISEE) program is one element of the NSF- 
sponsored Center for the Advancement of Engineering Education (CAEE). Its primary goal is to build a 
community of engineering education scholars who can think and work across disciplines with an ultimate 
aim of improving the engineering student experience. 

Three year-long institutes were hosted by three CAEE campuses each with a different theme and 
scholarship focus: University of Washington, 2004-2005, Classroom as Lab; Stanford University, 2005- 


Impacts studies all start with a 
concern, work towards a goal, and 
vary in terms of whether or not good 
information exists out there to use. 

culminating event. Each Institute cycle examined scholarship from a different angle. This paper focuses 
on the “Scholarship of Impact” framework from the 2005 Stanford Institute. 

Defining Scholarship of Impact 

Definitions for a “Scholarship of Impact” emerged during the 2005 Summer Summit at Stanford 
University. Participants and facilitators were given the task of defining impact and what would constitute 
an impact study. 

The ISEE team broadly defines impact as the measurement or evidence of change. Along with evaluation, 
research helps to define measures of impact that can be used to assess program changes (e.g., whether 
program goals were met). In turn, evaluation activities may inspire new theories and definitions of 
program impact. These examples illustrate the linked relationship between research and practice. Practical 
problems prompt and define research problems, and research problems seek to find answers to resolve 
practical problems. As the scope and complexity of the problem increases, measures of “impact potential” 
- the capacity to cause changes - become important for breaking down seemingly intractable goals into 
more realizable intermediate goals. 

Therefore, in the context of engineering education research, “impact studies” are distinguished from other 
kinds of research studies. While most studies seek to have theoretical and practical significance, impact 
studies seek to close the gap between research and practical problems. 


2006, Campus as Lab; and Howard University, 2006-2007, 
Nation as Lab. 

Each Institute cycle involved adapting and improving the 
model from the year before, recruiting Scholars, hosting them 
at a week-long Summer Summit kick-off, academic year 
activities to support Scholars conducting their studies, and a 


What Makes a Good Impact Study? 

The Stanford Scholars reached consensus on some features of good impact studies: 

• Intentional (with an explicit point of view) 

• Collaborative (includes diverse stakeholders) 

• Articulated at the start of the project 

• Made up of intermediate goals (with intermediate metrics of success) 

• Involve gathering information about the problem (drawing on existing information and collecting 
new information) 

• Involve identifying impact pathways (those who care, who can facilitate or block progress) 

• Meet the needs of the users (stakeholders) 

• Involve telling a compelling story for different audiences 

Impact studies all start with a concern, work towards a goal, and vary in terms of whether or not good 
information exists out there to use. 

Key questions to consider were defined and compiled in a "Scholarship of Impact" study plan worksheet 
used by Scholars. Through this process, big goals of each individual’s impact study were broken down 
into framing first steps, describing information needed, and identifying who was important in the impact 
pathway. 

An Example Impact Study 

A group at Stanford University has been examining the early undergraduate engineering experience as 
part of a shared 2005 Institute project with faculty, staff from the local Center for Teaching and Learning, 
and student Scholars. The group worked closely with the office of the Vice Provost for Undergraduate 
Education and the School of Engineering to investigate the early engineering education experience at 
Stanford. This involved examining the role that introductory engineering courses have in shaping the 
students' experience (for exploring engineering, providing exposure or breadth to various engineering 
fields or subspecialties, or providing depth in one disciplinary area). 

Data was collected on the School of Engineering introductory classes about student enrollment (who takes 
these courses and when do they take them), from academic transcripts (with collaboration with the 
university registrar and assistance from the CAEE Academic Pathways Study) as well as through faculty 
focus groups, benchmarking of other institutions, etc. 

Stakeholders (students, faculty, departments, university administrators and staff) were engaged early and 
often. Their particular responsibilities for the undergraduate student were mapped and change agents were 
specifically targeted among the university’s administration and faculty. Change opportunities at Stanford 
were identified that could positively impact student learning quality and satisfaction with the early 
engineering experience of undergraduate engineering majors. 

This example of “Scholarship of Impact” was enthusiastically supported by the Dean of Students in the 
School of Engineering and presented to a wider audience of deans, administrators, and curriculum 
committees. The research is spurring a review of the role of introductory engineering courses by the 
Undergraduate Engineering Council and a reform of this part of the curriculum is expected in the near 
term. This work is an example of the “Scholarship of Impact” framework in practice and illustrates what 
may be possible with this approach. It is hoped that this framework will be of interest and transferable to 
other engineering education research projects. 

www.engr.washington.edu/caee November 2007 


CAEE Research Brief 


2 


